Replicability and reproducibility of computational models has been somewhat understudied by “the replication movement.” In this paper, we draw on methodological studies into the replicability of psychological experiments and on the mechanistic account of explanation to analyze the functions of model replications and model reproductions in computational neuroscience. We contend that model replicability, or independent researchers' ability to obtain the same output using original code and data, and model reproducibility, or independent researchers' ability to recreate a model without original code, (...) serve different functions and fail for different reasons. This means that measures designed to improve model replicability may not enhance (and, in some cases, may actually damage) model reproducibility. We claim that although both are undesirable, low model reproducibility poses more of a threat to long-term scientific progress than low model replicability. In our opinion, low model reproducibility stems mostly from authors' omitting to provide crucial information in scientific papers and we stress that sharing all computer code and data is not a solution. Reports of computational studies should remain selective and include all and only relevant bits of code. (shrink)

According to the expertise defense, practitioners of the method of cases need not worry about findings that ordinary people’s philosophical intuitions depend on epistemically irrelevant factors. This is because, honed by years of training, the intuitions of professional philosophers likely surpass those of the folk. To investigate this, we conducted a controlled longitudinal study of a broad range of intuitions in undergraduate students of philosophy (n = 226), whose case judgments we sampled after each semester throughout their studies. Under the (...) assumption, made by proponents of the expertise defense, that formal training in philosophy gives rise to the kind of expertise that accounts for changes in the students’ responses to philosophically puzzling cases, our data suggest that the acquired cognitive skills only affect single case judgments at a time. There does not seem to exist either a general expertise that informs case judgments in all areas of philosophy, or an expertise specific to particular subfields. In fact, we argue that available evidence, including the results of cross-sectional research, is best explained in terms of differences in adopted beliefs about specific cases, rather than acquired cognitive skills. We also investigated whether individuals who choose to study philosophy have atypical intuitions compared to the general population and whether students whose intuitions are at odds with textbook consensus are more likely than others to drop out of the philosophy program. (shrink)

Most discussions of the reproducibility crisis focus on its epistemic aspect: the fact that the scientific community fails to follow some norms of scientific investigation, which leads to high rates of irreproducibility via a high rate of false positive findings. The purpose of this paper is to argue that there is a heretofore underappreciated and understudied dimension to the reproducibility crisis in experimental psychology and neuroscience that may prove to be at least as important as the epistemic dimension. This is (...) the communication dimension. The link between communication and reproducibility is immediate: independent investigators would not be able to recreate an experiment whose design or implementation were inadequately described. I exploit evidence of a replicability and reproducibility crisis in computational science, as well as research into quality of reporting to support the claim that a widespread failure to adhere to reporting standards, especially the norm of descriptive completeness, is an important contributing factor in the current reproducibility crisis in experimental psychology and neuroscience. (shrink)

We argue that Yarkoni's proposed solutions to the generalizability crisis are half-measures because he does not recognize that the crisis arises from investigators' underappreciation of the roles of theory in experimental research. Rather than embracing qualitative analysis, the research community should make an effort to develop better theories and work toward consistently incorporating theoretical results into experimental practice.

The paper presents two objections against Putnam’s Twin Earth argument, which was intended to secure semantic externalism. I first claim that Putnam’s reasoning rests on two assumptions and then try to show why these assumptions are contentious. The first objection is that, given what we know about science, it is unlikely that there are any natural-kind terms whose extension is codetermined by a small set of microstructures required by Putnam’s indexical account of extension determination. The second objection is that there (...) may not be a plausible concept of a speech community whose adoption would classify Oscar and Twin Oscar as members of different speech communities and, at the same time, render Oscar and Twin Oscar as being in the same psychological state. I contend that Putnam’s argument fails because both objections are justified. (shrink)